Fix renv restore in CI by pinning a dated PPM snapshot; add survRM2 - #613
Closed
MayaGans wants to merge 2 commits into
Closed
Fix renv restore in CI by pinning a dated PPM snapshot; add survRM2#613MayaGans wants to merge 2 commits into
MayaGans wants to merge 2 commits into
Conversation
CI has failed at "Set up renv" on every branch since ~24 July 2026, including pushes straight to main. The trigger was GitHub migrating ubuntu-latest to a new image: setup-renv keys its package cache on sessionInfo()$running, so the migration invalidated every cache at once. That exposed a latent problem. renv.lock pointed at packagemanager.posit.co/cran/latest, which only ever serves the *current* version of each package, and 154 of the 374 pinned versions were no longer current. With no binary available, renv fell back to compiling old source tarballs from the CRAN archive, which fails on the new runner. A warm cache had hidden this for months. Changes: - Pin the repository to a dated snapshot (cran/2026-08-14) and re-snapshot so every recorded version matches that date. Verified that all 365 repository-sourced records resolve on macOS arm64, Windows and Linux, and that all 170 compiled packages are served as prebuilt binaries for both noble and resolute. CI now compiles nothing. - Stop recording the 15 packages that ship with R (mgcv, Matrix, survival, lattice, ...). Their versions track the R build rather than CRAN, so no snapshot date can ever satisfy them; this is also what broke local installs on macOS. utils/quarto_check_pkg_dependencies.R already handles these having no hash. - Add survRM2 1.0-4 and its dependency muhaz, to unblock PSIAIMS#598. - Drop survMisc: CRAN archived it on 17 March 2026, it is no longer a survminer dependency, and it is only mentioned in prose. km.ci, KMsurv, plyr, reshape2 and rgl go with it as transitive dependencies that current versions no longer need. - Pin runs-on to ubuntu-24.04 in both workflows so a future image migration cannot silently invalidate the caches again. Note: 138 package versions change, so the Quarto freeze cache will be largely invalidated and most pages will re-execute on the first run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two notes in the "Setting up CAMIS with renv" section: - Check the `renv::snapshot()` diff before committing. Snapshot records the local environment, so it rewrites requirements.txt and the Python block of renv.lock from whichever Python environment is active, and sets the recorded R version to whatever R the contributor is running. An empty local venv has previously cut requirements.txt from 124 packages to one. - The package repository is pinned to a dated PPM snapshot, so installs get that date's version and packages published after it are unavailable until the date is moved forward. Explains why the pin exists (binaries on all three platforms) and why R-bundled packages are excluded via ignored.packages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's broken
Every CI run since about 24 July has failed at the "Set up renv" step — every
branch, including pushes straight to
main. So this isn't something #598, #608or #609 did. It broke on its own and it's been blocking everything since.
Why
Two things went wrong together.
renv.lock was asking for package versions that no longer exist. It pointed
at the Posit Package Manager "latest" address, which only ever offers the
newest version of each package. Our lockfile pinned 154 packages to older
versions than that. When renv can't get a ready-built copy, it falls back to
building the package from source code — and a lot of those old versions won't
build on a modern Linux machine.
But we never noticed, because of the cache. GitHub kept a saved copy of the
whole package library, built back when those versions were current, and every
run just reused it. Then GitHub swapped
ubuntu-latestto a newer Ubuntu imagein mid-July. The cache is tied to the OS version, so it was thrown away, and
suddenly every run had to install all 374 packages for real. That's when the
150-odd source builds actually had to happen, and they failed.
This is also why Abi's
ubuntu-22.04change in #609 didn't help — changing theUbuntu version just gives you a different empty cache, so you still face the
same failing builds. Nothing wrong with the idea, it just couldn't work alone.
And it's the same reason
renv::restore()has been failing on Macs locally.What this PR does
Points renv at a fixed date instead of "latest." The package manager keeps
dated archives, so
cran/2026-08-14gives us a snapshot where every version wewant is available pre-built for Linux, macOS and Windows. I then re-took the
snapshot so all our recorded versions match that date. CI now downloads
ready-made packages and compiles nothing.
Stops pinning the packages that come bundled with R. Things like
mgcv,Matrix,survivalandlatticeare installed as part of R itself, so theirversion depends on which R you have, not on CRAN. Pinning them can never work —
this is exactly what was failing on Macs, where the lockfile wanted mgcv 1.9-3
but R 4.5.3 ships 1.9-4. They're now excluded, and everyone just uses whatever
their R came with. The pre-render script already copes with this.
Adds survRM2 (and muhaz, which it needs) so #598 can finally go in.
Removes survMisc. CRAN archived it back in March,
survminerno longerneeds it, and our pages only mention it in the text — nothing actually calls it.
A few packages that were only there to support it (
km.ci,KMsurv,plyr,reshape2,rgl) drop out too.Pins the Ubuntu version in both workflows. Not because 24.04 is special, but
so that the next time GitHub moves
ubuntu-latest, it doesn't silently bin ourcache and catch us out again.
Adds notes to the contributor guide about the two things above that bite
people.
Adding a new package from now on gives you the version that was current on
14 August. If you need something newer, we move the date forward, which is a
change worth doing on its own rather than tacked onto a content PR.
Two things to keep an eye on afterwards
Someone needs to move the snapshot date forward every so often. Now that
we're pinned to 14 August 2026, we stay on those package versions until someone
deliberately bumps the date. That's the point - it's what makes restores
reproducible! But it does mean the gap between us and current CRAN grows
quietly in the background.
I'd suggest treating it as a routine maintenance job: move the date forward
every six months or so, ideally at the same time as an R version bump, and
always as its own PR rather than bolted onto a content change. This is more or
less what Christina used to do for us. Worth either putting a recurring
reminder somewhere or making it a standing agenda item, because it won't happen
if it depends on somebody remembering!
Worth Noting
Snapshots taken on a laptop pick up your local setup.
renv::snapshot()also rewrites
requirements.txtand the Python version from whatever Pythonenvironment you happen to have active. While putting this PR together it cut
requirements.txtfrom 124 packages down to 1, because my local Pythonenvironment is empty. I caught it and put it back, and it's the same thing that
affected #608.
I've added a note to the contributor guide for now, but doing snapshots in CI
instead would remove the trap altogether. Happy to set that up separately if
people think it's worth it??